"Recursive Self-Improvement and the World's Most Important Math
Problem." In 1965, I. J. Good suggested that smarter minds can make
themselves even smarter, leading to a runaway positive feedback that I. J. Good termed the "intelligence explosion". But how do you build an
Artificial Intelligence such that it remains stable and friendly through
the ascent to superintelligence? Eliezer Yudkowsky talks about the
implications of recursive self-improvement, and how it poses the most important math problem of this generation.